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ABSTRACT 

Evaluation can be crucial to help meet the needs and 
requirements of the teacher, students, and curriculum in distance 
education. Distance education usually encompasses a wide diversity of 
students, multi-site classrooms, little opportunity for interaction 
among students, and unfamiliar technology. Because of these 
uncertainties, evaluation of both courses and programs will provide 
teachers with information that can help make instruction effective. 
Formative evaluation takes place during all stages of instruction and 
is designed to promote revision of procedures and/or materials. 
Summative evaluation takes place at the end of instruction and is 
designed to assess the overall effectiveness of the instruction. A 
quantitative evaluation yields a breadth of information, and asks 
questions that can be statistically tabulated and analyzed. A 
qualitative evaluation gathers a wider range and depth of information 
using open-ended questions, interviews, and observations. A 
successful evaluation entails reviewing course goals and objectives, 
knowing students' needs, choosing the information to be gathered, 
collecting and analyzing data, and incorporating the evaluation 
findings in the revision. This brief guide begins with a statement on 
the importance and purpose of evaluation in distance education. It 
then presents guidelines for six steps in the evaluation process: (1) 
planning evaluation strategy; (2) reviewing course goals and 
objectives; (3) knowing students and their needs; (4) deciding what 
information should be gathered; (5) collecting and analyzing data; 
and (6) using the results to revise instruction. (DB) 
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EVALUATION— VITAL TO DISTANCE EDUCATION 



Distance education is an emerging field open to innovation, challenge, and development. 
As a distance educator, 

Vnn no longer have: 

• A traditional, familiar classroom 

• A relatively homogeneous group of students 

• Face-to-face feedback during class 

• Total control over the medium used in your teaching 

• Ease and convenience for otudent conferencing 
Ngvv, you have: 

• A multi-site "classroom" 

• A wide diversity of students 

• Learners who have little opportunity for interaction with fellow students 

• An unfamiliar technology that may fight you 

• Obstacles for student support. 

The purpose of evaluation... 

From an instructor's perspective, you evaluate to determine: 

• Whether the course objectives are being met. 

• What works, what doesn't. 

• How well the students are doing. 

• If and how the course can be improved. 

Fvaluation can he a kev to heln moat the needs and requirements of the instructor, 
c^a-qnts. and curriculum. 

Why evaluate... 

Course and program evaluation will: 

• Give you control in creating a better program. 

• Provide you with information to help you make instruction, materials, and 

processes as effective as possible. 

• Allow student involvement in helping them accomplish their objectives. 

• Help to determine whether or not growth and change are taking place. 

• Add to the information on effective distance delivered education. 



To think about... 

• How will I know if the students are getting the course content they need? 

• How effective am I as a facilitator, lecturer, counselor, evaluator, discussion leader? 

• is everyone comfortable using the equipment? 

• Are the assignments overwhelming or mundane? 

• Is there clarity of content? 

• Is class time well spent? 

• What student information do I want? Age, occupation, reasons for taking the class... 

• Will the evaluative information I gather enable me to make the course more interesting and 

relevant? 



INTRODUCTION 



Developing instruction is a critical part of the educational process. For instruction to be effective, it should be 
organized and well planned. Development of an effective program should include careful considers ion of the 
need for instruction, the audience, and the course content. Instruction delivered via telecommunication car. be 
even more complex and therefore best developed using a structured, systematic approach, such as the one 
presented here. 

This instructional development process can be useful for providing organization and direction for your instruc- 
tion It can also provide the flexibility needed to expand or shorten the process itself. Decisions to modify he 
process should be based on the needs of the learner, requirements for content, and constraints placed on the 
instructor. 

Most instructional development processes include four basic stages: Design, Development, Evaluation, and 
Revision. These are briefly described below in order of approach. 



DESIGN 

□ Determine the need for instruction. To begin, determine the need for instruction from a number 
of perspectives, such as: 



• How do you know the planned Instruction is needed? 

• Why does the audience need this instruction? 

• What factors led to this need for Instruction? 

• What past experiences indicate that the instruction you are planning could meet this need? 

• What are the differing views on issues such as content and delivery methods? 

Performing this assessment of need will help you develop your instruction to fill the gap between 
"What is . . ." and "What should be . . ." 

□ Analyze your audience. It is also important to look carefully at the learner group or audience 
whom the instruction is intended. Your analysis may include a number of perspectives, such as: 

• How large is the audience? 

• What ages, cultural backgrounds, and educational levels are included? 

• How familiar is the audience with various Instructional methods? 

• How will the audience use the Information presented? 

• What Is the audience level of interest and ability? 

The more thoroughly you understand the characteristics of each learner group, the more relevant the 
instruction can be. 

□ Establish instructional goals. Goals can now be established. Instructional goals , ar e usually 
broad, generalized statements of purpose based on the needs assessment and the audience 
analysis. 



DEVELOPMENT 



1. 



□ Develop specific objectives for each goal. Begin with a methodical division of your broad goals 
into a series of objectives or statements of accomplishment. Generally, have objectives build on each 
other, leading to the achievement of each goal. This step-by-step approach to developing objectives 
can help ensure that your topic is systematically examined and that major issues are covered in 
logical, sequential order. 

When developing objectives, have them: 

• Stated In precise terms describing exactly what should be done. 

• Written as descriptions of performance and stated In terms that can be measured. 

• Worded so that learning experiences can be developed to satisfy each objective. 

• Build into a learning hierarchy. 

□ Create a rough content outline. After developing objectives, create a rough outline of the content 
to be covered based on the audience analysis, instructional goals, objectives, and your understand- 
ing of the course content. 

□ Review existing materials. Next, locate and review content-related print and non-print materials 
currently in existence. It is often more cost and time efficient to use or adapt existing materials than to 
develop new ones. However, avoid using materials or media just because they are available. 



□ Organize/develop content to meet objectives. Reorganize, revise, and refine your rough 
outline to correspond to the order and content needs indicated by your objectives. Next, develop the 
content required to meet your objectives. 



□ Select/develop materials and methods. Base your selection and development of materials and 
delivery methods on the following criteria: 



• Content and objectives. 

• Audie we characteristics and experience. 

• Time, money, and personnel availability/constraints. 



Notes: 



EVALUATION 



□ Rwiew Goals and Objectives. The fundamental purpose of evaluation is to determine if your 
instructional methods and materials are accomplishing the goals and objectives you developed. 
Implementing your instruction represents the first real test of what you have developed. Try to pre-test 
your instruction on a small scale prior to actual use. If this is not possible, your first actual use will also 
serve as the "field test" for determining effectiveness. 



□ Develop an evaluation strategy. Plan how and when to evaluate the effectiveness of your instruc- 
tion. Evaluation throughout the teaching process (formative) can be used to revise instruction before 
the course Is completed. Evaluation after instruction is completed (summative) provides a data base 
for planning future courses. If possible, it is best to use a combination of formative and summative 
evaluation methocis throughout the instructional process. Consider evaluating the following: 

• Audience attitude and performance. 

• Strengths and weaknesses of program. 

» Effectiveness of teaching techniques and delivery systems. 

• Instructor effectiveness. 

• Degree to which course objectives were met. 

□ Collect and analyze evaluation data. Following implementation of your course/materials, collect 
your evaluation data. Carefully analyze these results to identify gaps or weaknesses in your instruc- 
tional process. It is equally important to identify strengths and successes. 



□ Recommend revisions based on analysis. Results of your evaluation analysis should indicate 
which elements of your instruction should be changed or eliminated and which should be retained. 
They provide a "springboard" from which to develop a revision plan. 



REVISION 



□ Develop and implement a revision plan. A certain amount of revision is to be expected as a 
result of your evaluation efforts. Typically, however, this revision is worth the effort and should result in 
more effective instruction. Revisions will require returning to the Design, Development, or Evaluation 
stages to make changes in the ongoing process of Instructional Development. 



i 



/ 



( ) 



ri 0 f' 




FOR MORE INFORMATldN: 

Statewide Distance Education 
University of Alaska System 
3890 University Lake Drive 
Anchorage, Alaska 9.9508 



